To produce a color sensitive image sensor, the pixels in the image sensor's pixel array are made to have different sensitivities to different frequencies of incident light, with the chosen frequencies corresponding to particular colors. A common way of achieving this is to overlay the image sensor's pixel array with a color filter array so that each pixel is overlaid with a material with specific transmission characteristics so that the pixel can be thought of as being responsive to a particular color. Usually one color is overlaid per pixel. As a short-hand notation, a red pixel is referred to as a pixel that is sensitive to incident light having a frequency in the red part of the visible spectrum of light, with similar notations used for other colors. The specific frequency responses will vary according to the type of material used, among other factors.
There are various well known patterns to use when depositing the color filter material. For each pattern, it is possible and common to separate the output of the sensor into groups of component colors. Each of these groups forms a color plane. The number of color planes and also how they are mapped onto the sensor is dependent on the color pattern used.
Unless specifically indicated otherwise, the term neighboring pixels as used herein is taken to mean that the two pixels are adjacent in a given color plane to which they both belong, rather than necessarily being physically adjacent.
To illustrate this point, FIGS. 3-6 illustrate some example color patterns. FIG. 3 shows a monochrome sensor. Each block represents a pixel which outputs a chrominance value, Y. The coordinates of each pixel are also shown. Here, neighboring pixels correspond to physically adjacent pixels, so for example, Y(3,2) is a neighbor of Y(4,2). To avoid any doubt, the color plane with respect to a monochrome sensor such as this is taken to correspond to the physical array.
FIG. 4 illustrates the common Bayer pattern, which comprises red, green and blue sensitive pixels, labeled R, G, B respectively, and with the color plane coordinates shown. As can be seen, there are twice as many green pixels as either the red or blue. Because the human eye is more sensitive to spatial information in the green channel, the sensor produces an image which appears more detailed to the human eye than other color pattern schemes. The top FIG. shows the entire array, while the lower figures show each color plane. Hence, pixels which are physical neighbors on the image sensor plane are no longer neighbors when separated into color planes. For example, R(2,2) and G(2,3) are physical neighbors but not after separation into color planes.
Also, pixels which are not neighbors on the image plane will become neighbors in color space. For example, G(2,2) and G(3,2) are not physically adjacent on the image sensor, but are neighbors after separation into color planes.
Various manufacturers use four color filters in a 2×2 grid. One of the greens is replaced with a similar, but slightly different color. This increases the color performance of the system but with only a marginal degradation in luminance detail. This technique is illustrated in FIG. 5, with the fourth color labeled as “T”.
As well has 2×2 structures of color, it is also possible to have stripes of color. An example is shown in FIG. 6. This stripe pattern is popular with CRT manufacturers as it increases the tolerance of the electron gun alignment, but is not commonly used on CMOS/CCD sensors.
Image sensors commonly have defective pixels. If the number of these is small, their effect can be minimized electronically. This can be done for isolated pixel defects by use of a filtering algorithm during use of the image sensor, i.e., on the fly (reference EP1003332A). Another such technique (reference EP05250101.2) uses a scythe filter which operates over a 3×3 pixel kernel and sorts the pixels into an order based on their magnitude. Essentially, defective pixels are considered to be stuck at either dark or light, and therefore will lie at either end of a magnitude sorted list. These can then be replaced with the nearest value pixel.
However, in the case of a couplet defect, both pixels are within the kernel of operation of the filtering algorithm, and the defects are not corrected, as the replacement pixel is of the same magnitude as the identified defective pixel.